Method of providing user specific interaction using device and digital television(dtv), the dtv, and the user device

ABSTRACT

An interactive method includes displaying image content received through a television (TV) network, identifying an object of interest of a user among a plurality of regions or a plurality of objects included in the image content, and providing additional information corresponding to the object of interest.

CROSS-REFERENCE TO RELATED APPLICATIONS

This present application is a continuation of U.S. patent applicationSer. No. 14/016,573 filed Sep. 3, 2013, which claims the prioritybenefit of Korean Patent Application No. 10-2013-0017974, filed on Feb.20, 2013, in the Korean Intellectual Property Office, the disclosure ofwhich is incorporated herein by reference.

BACKGROUND

1. Field

The following description relates to a method of providing user specificinteraction using a digital television (DTV), the DTV, and a userdevice.

2. Description of the Related Art

A digital television (DTV) removes noise such as interference by otherradio waves, by converting analog signals into digital signals andcontrolling the digital signals by a computer circuit, thereby providingincreased image quality. In addition, the DTV is capable of selectingvarious information and programs as well as performing high levelautomatic control. Also, the DTV may reproduce a variety of high endimages such as a picture in picture (PIP).

SUMMARY

The foregoing and/or other aspects are achieved by providing a method ofproviding user specific interaction, the method including receivingimage content through a television (TV) network, displaying the imagecontent, identifying an object of interest of a user among a pluralityof regions or a plurality of objects included in the image content, andproviding additional information corresponding to the object ofinterest.

The identifying may include tracking sight of the user using aphotographing device, and identifying an object tracked by the sight ofthe user, among a plurality of regions or a plurality of objectsincluded in the image content, as the object of interest of the user.

The method may further include receiving signals from a user device forthe user, wherein the identifying may include identifying the object ofinterest using the signals received from the user device.

The identifying of the object of interest using the signals may includeanalyzing the signals received from the user device, determining a scenein which the signals are changed in the image content as a result of theanalysis, and identifying a region or object included in the scene inwhich the signals are changed as the object of interest.

The signals received from the user device may include at least one of avital signal of the user detected by the user device and an externalforce applied to the user device.

The method may further include searching for additional informationcorresponding to the object of interest.

The additional information corresponding to the object of interest mayinclude at least one of information providing sensory stimuluscorresponding to the object of interest, multinational languagesubtitles related to the image content that includes the object ofinterest, translation of a finger language, a Braille type, oscillation,audio supply, audio conversion, user specific sound, and information forenlargement of subtitle font.

The information providing sensory stimulus may include at least one ofhaptic information stimulating a haptic sense, sound informationstimulating an auditory sense corresponding to the object of interest,display information stimulating a visual sense corresponding to theobject of interest, smell information stimulating an olfactory sensecorresponding to the object of interest, and taste informationstimulating a taste sense corresponding to the object of interest.

The foregoing and/or other aspects are also achieved by providing amethod of providing user specific interaction, the method includingreceiving image content through a TV network, displaying the imagecontent, receiving an input of selection with respect to an object ofinterest of a user among a plurality of regions and a plurality ofobjects included in the image content, allowing interaction of the userwith respect to the object of interest, and providing additionalinformation corresponding to the object of interest in response to theinteraction of the user.

The interaction may include at least one of a vital signal of the userdetected by a user device and an external force applied to the userdevice.

The method may further include analyzing the interaction of the userwith respect to the object of interest.

The method may include searching for additional informationcorresponding to emotion or request of the user recognized from theinteraction of the user based on a result of the analyzing.

The providing of the additional information may include providing theadditional information being found as the additional informationcorresponding to the object of interest.

The additional information corresponding to the object of interest mayinclude at least one of information providing sensory stimuluscorresponding to the object of interest, multinational languagesubtitles related to the image content that includes the object ofinterest, translation of a finger language, a Braille type, oscillation,audio supply, audio conversion, user specific sound, and information forenlargement of subtitle font.

The foregoing and/or other aspects are also achieved by providing amethod of providing user specific interaction, the method includingdetecting an event generated in a user device in response to imagecontent displayed on a DTV, identifying an object of interest of a useramong a plurality of regions or a plurality of objects included in theimage content based on the event, and transmitting information on theobject of interest so as to be provided with additional informationcorresponding to the object of interest.

The identifying may include identifying a region or object included inthe image content of a time point when the event is detected, among theplurality of regions or objects, as the object of interest.

The event generated in the user device may include at least one of achange in a vital signal of the user detected by the user device and achange in an external force applied to the user device.

The method may further include receiving additional informationcorresponding to the object of interest from the DTV, and realizing theadditional information corresponding to the object of interest.

The method may further include transmitting signals including the eventgenerated in the user device to the DTV.

The foregoing and/or other aspects are also achieved by providing adigital television (DTV) including a communication unit to receive imagecontent through a TV network, a display unit to display the imagecontent, and a control unit to identify an object of interest of a useramong a plurality of regions or a plurality of objects included in theimage content and provide additional information corresponding to theobject of interest.

The control unit may include an analysis unit to analyze signalsreceived from a user device, and a determination unit to determine ascene in which the signals are changed among a plurality of regions or aplurality of objects included in the image content as a result of theanalyzing, wherein the DTV may identify a region or object included inthe scene in which the signals are changed as the object of interest.

The control unit may further include a search unit to search foradditional information corresponding to the object of interest.

The foregoing and/or other aspects are also achieved by providing a DTVincluding a communication unit to receive image content through a TVnetwork, a display unit to display the image content, and a control unitto receive an input of selection with respect to an object of interestof a user among a plurality of regions or a plurality of objectsincluded in the image content and provide additional informationcorresponding to the object of interest in response to interaction ofthe user.

The foregoing and/or other aspects are also achieved by providing a userdevice providing user specific interaction, the user device including adetection unit to detect an event generated in the user device inresponse to image content displayed on a DTV, an identifying unit toidentify an object of interest of a user among a plurality of regions ora plurality of objects included in the image content based on the event,and a communication unit to transmit information on the object ofinterest or signals including the event generated in the user device soas to be provided with additional information corresponding to theobject of interest.

The user device may further include a realization unit to realize theadditional information corresponding to the object of interest, whereinthe additional information is received from the DTV displaying the imagecontent.

Additional aspects, features, and/or advantages of example embodimentswill be set forth in part in the description which follows and, in part,will be apparent from the description, or may be learned by practice ofthe disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects and advantages will become apparent and morereadily appreciated from the following description of the exampleembodiments, taken in conjunction with the accompanying drawings ofwhich:

FIG. 1 illustrates a configuration of a system performing a method ofproviding user specific interaction according to example embodiments;

FIG. 2 illustrates a method of providing user specific interaction by adigital television (DTV) according to example embodiments;

FIG. 3 illustrates a method of identifying an object of interest of auser, according to example embodiments;

FIG. 4 illustrates a method of identifying an object of interest of auser, according to example embodiments;

FIG. 5 illustrates a method of providing user specific interaction by aDTV, according to example embodiments;

FIG. 6 illustrates examples of various user specific interactionsprovided between a DTV and user devices, according to exampleembodiments;

FIG. 7 illustrates various examples of interactions of a user allowedwith respect to an object of interest of the user, by a method ofproviding user specific interaction according to example embodiments;

FIG. 8 illustrates various examples of additional informationcorresponding to an object of interest which is provided to a userdevice by a method of providing user specific interaction according toexample embodiments;

FIG. 9 illustrates a method of giving opinions between user devices withrespect to image content provided by a method of providing user specificinteraction according to example embodiments;

FIG. 10 illustrates a method of providing user specific interaction by auser device, according to example embodiments;

FIG. 11 illustrates a DTV providing user specific interaction, accordingto example embodiments; and

FIG. 12 illustrates a DTV providing user specific interaction, accordingto example embodiments;

FIG. 13 illustrates a user device providing user specific interaction,according to example embodiments; and

FIG. 14 illustrates a system including a DTV and a plurality of userdevices which provide user specific interaction, according to exampleembodiments.

DETAILED DESCRIPTION

Reference will now be made in detail to example embodiments, examples ofwhich are illustrated in the accompanying drawings, wherein likereference numerals refer to like elements throughout. Exampleembodiments are described below to explain the present disclosure byreferring to the figures.

FIG. 1 illustrates a configuration of a system performing a method ofproviding user specific interaction according to example embodiments.

Referring to FIG. 1, in the system in which a digital television (DTV)110 and a plurality of user devices, such as user devices 120, 130, and140, for example, are combined, when a plurality of users appreciate thesame content provided through the DTV 110, different user specificinteractions may be provided to the user devices 120, 130, and 140.

Here, the DTV 110 and the user devices 120, 130, and 140 may beconnected to one another through a wired or wireless network 50.

The DTV 110 may analyze signals of the user devices 120, 130, and 140,and provide additional information corresponding to the analyzed signalsto the user devices 120, 130, and 140. Here, when the additionalinformation corresponding to the analyzed signals require conversion,the DTV 110 may convert the additional information to be appropriate forthe user devices 120, 130, and 140 or for the user, and provide theadditional information.

The user devices 120, 130, and 140 may receive an input of an object ofinterest of the user or preference information related to image contentfrom the user, or receive and provide to the DTV 110 informationaccording to a force applied to the user devices 120, 130, and 140 or avital signal of the user.

In addition, the user devices 120, 130, and 140 may realize or reproducethe additional information corresponding to the object of interestprovided from the DTV 110.

Thus, according to the example embodiments, when a plurality of usersappreciate the image content, such as drama, movie, or commercials, forexample, provided by the DTV 110, the user devices may sense and analyzethe vital signal of the users or the force applied to the user devices120, 130, and 140 using the user devices 120, 130, and 140,respectively. Also, the user devices 120, 130, and 140 may exchangeinteractions with the DTV 110 using a result of the sensing andanalyzing. Here, the interactions exchanged between the user devices120, 130, and 140 and the DTV 110 may be user specific interactionrelated to the image content displayed by the DTV 110.

Therefore, the DTV 110 may provide various information and direct orindirect experiences related to an object with which the user isinterested when watching the image content such as commercials, movies,and dramas, for example.

Furthermore, when the users select the preference information of theirown using the user devices 120, 130, and 140, the DTV 110 may provideadditional information corresponding to the preference information ofthe users among the image content being reproduced, to the user devices120, 130, and 140.

Thus, according to the example embodiments, interaction with respect tothe content watched by the plurality of users may be enabled through theuser devices 120, 130, and 140. Therefore, immersion and amusement withrespect to the content may be increased, as a user specific service orfeedback stimulating senses according to users are received. Moreover,even disabled people may enjoy the same content together with otherpeople without a particular device dedicated for the disabled people.

FIG. 2 illustrates a flowchart of a method of providing user specificinteraction by a DTV, according to example embodiments.

Referring to FIG. 2, the DTV according to the example embodimentsidentifies or estimates an object of interest of a user and providesadditional information corresponding to the object of interest.

The DTV may receive image content through a TV network in operation 210and display the image content in operation 220. The ‘TV network’ may beunderstood to include a broadcasting network, and the Internet, forexample.

The DTV may identify the object of interest among a plurality of regionsor a plurality of objects included in the image content, in operation230. The ‘object of interest’ may refer to an object in which the useris interested, or an object with respect to which the user wants to beprovided with various additional information, that is, the user specificinteraction, besides the image content through the DTV.

The method of identifying the object of interest by the DTV in operation230 will be described in detail with reference to FIGS. 3 and 4.

In operation 240, the DTV may provide the additional informationcorresponding to the object of interest identified in operation 230.Depending on embodiments, the DTV may search for the additionalinformation corresponding to the object of interest and provide theadditional information to user devices.

Here, the user devices may include a smart phone, a mobile personalcomputer (PC), a wearable user device such as glasses, and a flexibledisplay, for example.

The additional information corresponding to the object of interest mayinclude the information providing sensory stimulus corresponding to theobject of interest, multinational language subtitles related to theimage content that includes the object of interest, translation of afinger language, a Braille type, oscillation, audio supply, audioconversion, user specific sound, information for enlargement of subtitlefont, and the like.

The information providing sensory stimulus corresponding to the objectof interest may include at least one of haptic information stimulating ahaptic sense, sound information stimulating an auditory sensecorresponding to the object of interest, display information stimulatinga visual sense corresponding to the object of interest, smellinformation stimulating an olfactory sense corresponding to the objectof interest, and taste information stimulating a taste sensecorresponding to the object of interest.

FIG. 3 illustrates a method of identifying an object of interest of auser, according to example embodiments.

Referring to FIG. 3, the method 230 of identifying the object ofinterest by a DTV will be described.

The DTV may track a sight, or a gaze, of the user using a photographingdevice in operation 310. Here, the photographing device may track thesight of the user, that is, a movement of a black part of a pupil of theuser. For example, as shown in 605 of FIG. 6 that will be describedlater, the photographing device may be disposed at an upper end of theDTV or another position. The photographing device may be single orplural.

The DTV may identify an object tracked by the sight of the user amongthe plurality of regions or objects included in the image content, asthe object of interest of the user, in operation 320.

For example, when the sight of the user, among the plurality of regionsor objects included in the image content displayed on a screen of FIG.6, moves along a movement of a heroin, the DTV may identify the heroineas the object of interest. When the sight of the user stays at a carincluded in the image content of FIG. 6, the DTV may identify the car asthe object of interest.

Thus, the object of interest may include anything of the regions orobjects included in the image content, such as a person, an object, anda background, for example.

FIG. 4 illustrates a method of identifying an object of interest of auser, according to example embodiments.

Referring to FIG. 4, a DTV according to the example embodimentsidentifies the object of interest using signals received from a userdevice.

The DTV may receive the signals from the user device for the user, inoperation 410. The signals may include at least one of a vital signal ofthe user detected by the user device and an external force applied tothe user device.

Here, the vital signal may include a biorhythm signal, a bodytemperature, a pulse, brain waves, an eye movement, and the like. Theexternal force applied to the user device may include deformation, adegree of bending, a pressure, a position, rotation of the user device,and a pressure and static electricity or magnetic field detected by atouch display of the user device.

In addition, the signals received from the user device may include asignal related to a motion or gesture of the user, an audio signal, andthe like.

In operation 420, the DTV may analyze the signals received from the userdevice.

In operation 430, the DTV may determine a scene in which a change ofsignal occurs in the image content as a result of the analysis ofoperation 420.

In operation 440, the DTV may identify a region or object included inthe scene in which the change of signal occurs, as the object ofinterest.

FIG. 5 illustrates a method of providing user specific interaction by aDTV, according to example embodiments.

Referring to FIG. 5, the DTV according to example embodiments analyzeuser interaction with respect to the object of interest directlyselected by the user, accordingly providing additional information.

The DTV may receive an image content image through a TV network inoperation 510, and display the image content in operation 520. Asaforementioned, the TV network may include both a broadcasting networkand the Internet.

In operation 530, the DTV may be input with a selection with respect tothe object of interest among a plurality of regions or a plurality ofobjects included in the image content.

In operation 540, the DTV may allow an interaction by the user withrespect to the object of interest selected in operation 530. Here, theinteraction of the user may include a vital signal of the user detectedby a user device and an external force applied to the user device.

In response to the interaction of operation 540, the DTV may provide theuser device with the additional information corresponding to the objectof interest.

In operation 550, the DTV may analyze the interaction of the user withrespect to the object of interest.

In operation 560, based on a result of the analysis of operation 550,the DTV may search for additional information corresponding to emotionor request of the user that is recognized from the interaction of theuser.

In operation 570, the DTV may provide the user device with theadditional information found in operation 560 as additional informationcorresponding to the object of interest.

As aforementioned, the additional information corresponding to theobject of interest may include information providing sensory stimuluscorresponding to the object of interest, multinational languagesubtitles related to the image content that includes the object ofinterest, translation of a finger language, a Braille type, oscillation,audio supply, audio conversion, user specific sound, information forenlargement of subtitle font, and the like.

FIG. 6 illustrates examples of various user specific interactionsprovided between a DTV 610 and user devices 630, 650, and 670, accordingto example embodiments.

Referring to FIG. 6, when a plurality of user devices 630, 650, and 670appreciate the same image content, various user specific interactionsmay be provided to the user devices 630, 650, and 670 by interactionbetween the DTV 610 and the user devices 630, 650, and 670.

The DTV 610 or the user devices 630, 650, and 670 may recognize acharacter who the user likes, by analyzing vital signals detected as theuser directly selects the character from a scene of the image content oras the user contacts his or her body part, such as a hand or face, forexample, with the user devices 630, 650, and 670.

The user devices 630, 650, and 670 may identify the object of interestby analyzing a force applied to the user devices 630, 650, and 670. Theuser devices 630, 650, and 670 may analyze information on the object ofinterest, thereby providing the additional information corresponding tothe object of interest. In addition, the user devices 630, 650, and 670may provide the information to the DTV 610 so that the DTV 610 mayprovide the user devices 630, 650, and 670 with the additionalinformation corresponding to the object of interest according to aresult of the analysis.

The DTV 610 may track a sight of the user using cameras 605 attached tothe DTV 610, thereby identifying the object of interest, such as acharacter, or an object, for example, corresponding to the sight of theuser. Although FIG. 6 illustrates three cameras 605, the disclosure isnot limited thereto. For example, one or two cameras may be used, ormore than three cameras may be used.

After the object of interest of the user is identified, the user may beprovided with emotions of the character the user likes in the imagecontent through additional information through the user devices 630,650, and 670. Here, the additional information may be provided as theuser specific interaction in the form of haptic feedback, display,sound, oscillation, or wind, for example.

For example, it may be presumed that a drama, as the image content,displayed on the DTV 610 includes a scene in which a hero appears in anice car in front of a heroine.

When the user of the user device 630 likes the heroine and watches amovement of the heroine, the cameras 605 mounted to the DTV 610 maytrack the sight of the user of the user device 630 and identify theheroine as the object of interest.

When the heroin and the hero like each other in the drama, the user ofthe user device 630 may be provided with the emotion of the heroine,such as a flutter, with respect to the hero through the user device 630.

Here, the flutter of the heroine with respect to the hero may beexpressed by additional information corresponding to the heroine, who isthe object of interest. For example, the flutter of the heroine may beexpressed by small and large heart shapes displayed on the screen of theuser device 630, or small and large oscillations of the user device 630.

When the user of the user device 650 is interested in the car of thehero and continues staring at the car, the cameras 605 may track thesight of the user of the user device 650, thereby identifying the car asthe object of interest. Here, it is presumed that the DTV 610 providesthe user device 650 with sales information or commercial informationrelated to the car as the additional information corresponding to thecar, that is, the object of interest.

Therefore, sales prices according to models of the car and otherinformation may be provided on the screen of the user device 650.

In this case, when color change, headlight control, engine sound, andthe like are provided as the additional information corresponding to thecar, that is, the object of interest, the user may operate the car orchange color of the car using the screen or buttons of the user device650.

The user of the user device 670 may be interested in a backgroundappearing in the image content.

Besides the above examples, when a hero or heroine the user likesappears in the drama, the user may give a stimulus to the user device,for example, apply a pressure by grasping the user device hard, draw aheart shape on the screen of the user device, wipe the user device, orpet the user device. Therefore, the user device may extract and showonly scenes given the stimulus. The user device may find and show othercontent in which the hero shown in the scenes appears.

In addition, when a spectacular landscape is shown in the image content,the user devices 630, 650, and 670 may analyze the vital signal of theuser or analyze a force applied by the user, for example, a pressurelevel and a shape of drawing. In addition, the user devices 630, 650,and 670 may make the user feel as if the user became the hero or heroineand were in the landscape in the scene given the corresponding stimulus,using a result of the analysis. Here, the corresponding stimulus mayinclude a change in the vital signal, or a change in the force appliedto the user device, for example.

For example, when the image content shows a scene showing a cool valley,the user devices 630, 650, and 670 may provide the user with a coolfeeling as if the user has soaked her hands in the valley water, ripplesof the valley water around the hands of the user, water sound along amovement of the hand, and the like.

The cool feeling may be provided by dropping a temperature of the userdevice. The ripples may be provided through haptic feedback byoscillating the user device periodically. The water sound may beprovided through a sound effect, that is, auditory feedback.

When the image content shows a scene of walking in an autumn scenerywith fallen yellow maple leaves, the user may be provided with the soundof stepping on the fallen leaves through auditory feedback, a feel oftouching the fallen leaves through haptic feedback, a change of thefallen leaves being moved by a hand through visual feedback and auditoryfeedback.

Thus, the user devices 630, 650, and 670 may convert contents marked ina script included in the image content such as a movie and a drama intothe information stimulating the senses, for example, haptic,oscillation, and the like, thereby giving feedback to the user.

Furthermore, the user devices 630, 650, and 670 may transmit acorresponding interaction to the user to provide senses, such as visual,haptic, taste, olfactory, and auditory senses of when he or she directlyoperates an object appearing in a commercial, a movie, and a drama.

For example, the user may be fed back with the sensory stimulus for thevisual, haptic, taste, olfactory, and auditory senses as if the he orshe were experiencing a person, object, animal, or landscape interestedby the user in the commercial, movie, and drama.

An example of providing feedback related to various senses including thevisual, auditory, haptic, and the like by the user devices 630, 650, and670 will be described as follows.

When a car is shown in the image content, the user devices 630, 650, and670 may display a 3-dimensional (3D) model of the car on the screen andmay generate interaction as the user pushes a button, thereby providingthe user with an experience of directly manipulating and driving thecar. Here, the interaction may include operations of sounding a hornwhen the user pushes a button corresponding to Klaxon in the userdevice, turning on a light when the user pushes a button correspondingto a headlight, generating trembles of when starting the car, and thelike. For example, audio, 3D model rendering, and augmented reality (AR)technologies may be used to generate such operations.

Depending on embodiments, the DTV may exchange signals with a massagechair or a robot cleaner being at home and then the massage chair or therobot cleaner may function as a driver seat of the car. That is, whenthe user sits on the massage chair or the robot cleaner, the user may beprovided with a feel of riding a car.

Various other types of feedback will be described in detail.

As to the visual feedback, when the user likes clothes a character inthe image content wears, the user device may display the user wearingthe clothes of the character. Also, the user wearing the clothes andmoving may be shown on the DTV. Here, the AR technology may be used.

As to the haptic feedback, when the image content shows an electric fanor air-conditioner commercial, the user device may blow air and delivera cool feeling to the user. When the image content shows a cold drink,the user may be provided a cold feeling by dropping the temperature ofthe user device.

As to the olfactory feedback, when the image content shows a pineymountain scene, the user device may deliver a natural smell of pineneedles or pine resin that the user remembers to the user. In detail,the user device may directly generate the natural smell or may searchfor the natural smell the user remembers from a memory of the user andtransmit a stimulus to the user's brain, so that the user feels as if heor she were smelling the natural smell in person. In addition, when theimage content displays coffee, the user device may deliver a coffeesmell to the user in the same manner.

As to the taste feedback, when the image content shows chocolate, theuser device may deliver sweetness to the user as if the user were eatingchocolate. In this case, the user device may search for an experience ofthe user of eating chocolate from brain waves or the memory, andtransmit a corresponding signal to the user's brain, so that the usermay feel an experience of eating chocolate.

Random olfactory and taste feedback may provide the user with a stimuluscorresponding to an experience of an olfactory stimulus or a tastestimulus of the past.

When a person receives a sensory stimulus, particular cells of ahippocampus of a brain are activated and store the stimulus in acerebral cortex. Therefore, the person may recall the stimulus whengiven the same stimulus. Accordingly, the example embodiments mayprovide types of feedback that stimulate the senses using the experienceabout the stimulus.

For example, when the person smells coffee by his or her nose, theparticular cells of the hippocampus may be activated and store acorresponding stimulus in the cerebral cortex. Reversely, when thecerebral cortex is given a stimulus same as when the user smells coffee,the user may receive the olfactory feedback, that is, a feeling as ifthe user actually smelt coffee.

When the user eats sweet chocolate, the particular cells of thehippocampus may be activated and store a corresponding stimulus in thecerebral cortex. Reversely, when the cerebral cortex is given a tastestimulus the same as when the user eats chocolate, the user may receivethe olfactory feedback, that is, a feeling as if the user actually atechocolate.

Here, activation of the particular cells in the hippocampus may berecognized by a camera, a rheometer, and the like and a stimuluscorresponding to signals detected by those devices may be analyzed anddetermined. Next, the user device may find corresponding stimulusinformation and give a feedback to the user and may visually show theprocess through the AR.

Also, depending on embodiments, the user may directly purchase goodsinterested by the user, among objects showing on the image content orcommercials, by applying a predetermined force to the user device.

FIG. 7 illustrates various examples of interactions of a user allowedwith respect to an object of interest of the user, by a method ofproviding user specific interaction according to example embodiments.

Referring to FIG. 7, image content 710 is reproduced through a DTV.

Here, a heroine who the user likes, called Olive, is shown in the imagecontent 710, the user may stand his or her hand on a side and scans thescreen of the user device as shown in 730. In this case, through aspecific motion of the user, which is the screen scanning in thisexample, the user device may store a scene including Olive currentlyshowing on the screen.

When a hero the user likes, called Tom, is showing in the image content710, the user may store a scene including Tom by bending a flexibledisplay functioning as the user device as shown in 750.

Furthermore, the user may become the hero, Tom, and store only scenes inwhich Tom feels love with Olive. In this case, the user may store thescenes corresponding to a heart shape representing the love emotion.Therefore, the user may retrieve the scenes by drawing the heart shapeon the screen of the user device as shown in 770.

Moreover, the user may give other stimuli to the user device, forexample, by grasping the user device hard to apply a pressure, wipingthe screen of the user device, and petting the user device. Throughthis, the user device may extract and show only scenes given thestimulus. The user device may find and show other contents in which thehero shown in the scenes appears.

FIG. 8 illustrates various examples of additional informationcorresponding to an object of interest which is provided to a userdevice by a method of providing user specific interaction according toexample embodiments.

Referring to FIG. 8, a movie 810 is reproduced through a screen of a DTVwatched by a plurality of users including a first user, a second user, athird user, and a fourth user. Here, when a scene of the movie 810 isthe object of interest of the users, different additional information830, 850, and 870 corresponding to the object of interest may beprovided to user devices of the respective users.

Here, the user device of the first user may be provided with a fingerlanguage translation service as additional information forhearing-impaired people corresponding to the scene of the movie 810 asshown in 830.

The user device of the second user may be provided with an Englishsubtitle as additional information for a foreign language translationdisplay as shown in 850.

The user device of the third user may be provided with an audio service,a Braille type, oscillation, and the like as additional information forthe visually handicapped as shown in 870.

In addition, corresponding to the scene of the movie 810, a languagetranslation service may be provided as the additional information. Forexample, the first user may be provided with a Korean subtitle, thesecond user may be provided with an English subtitle, and the third usermay be provided with a French subtitle.

Besides, according to the example embodiments, an age-based service maybe provided. For example, children may be provided with a dubbingservice that provides audio of a translated language. Elderly people maybe provided with additional information that increases sound volume orfont size.

A customized audio service may be provided through an earphone or bydifferent sounds according to positions of each user by applying a soundfocusing technology that uses ultrasonic waves.

In this case, contents marked in a script included in the content suchas a movie and a drama may be translated by an operator or convertedinto audio, thereby giving feedback to the user.

Thus, various types of information corresponding to the object ofinterest may be provided to the user device according to the exampleembodiments.

FIG. 9 illustrates a method of giving opinions between user devices withrespect to image content provided by a method of providing user specificinteraction according to example embodiments.

Referring to FIG. 9, the user device may generate a space, for example acyber space or a chatting room, for enabling users who are watching thesame channel or content communicate about objects including persons orobjects included in the content.

In operation 910, a user may designate an opponent user device tocommunicate through a chatting room with the users watching the samechannel or content. In addition, the user may draw a user opinion withrespect to the content using an application such as a short messageservice (SMS) in operation 920, and transmit the user opinion to theopponent user in operation 930.

Here, for example, the user opinion may usually relate to a character orclothes of the character.

The user may receive the user opinion of the opponent user in operation940, and reproduce content of the user opinion of the opponent user inoperation 950.

Thus, according to the example embodiments, users watching the samecontent may produce a chatting room and communicate about objects suchas a character, accessories of the character, places, buildings, and thelike, shown in the content such as a movie. In addition, users watchingthe same channel or content may produce a chatting room and communicateabout a character or clothes of the character using an application suchas the SMS.

FIG. 10 illustrates a method of providing user specific interaction by auser device, according to example embodiments.

Referring to FIG. 10, in operation 1010, the user device may detect anevent generated in the user device in response to image content beingdisplayed on a DTV.

The event may include at least one of a change in a vital signal of theuser detected by the user device and a change in an external forceapplied to the user device. For example, the user device may detect thechange in the vital signal through a device measuring a vital signal,such as a heart rate monitor (HRM) installed inside or outside the userdevice.

The change in the external force applied to the user device may includedeformation such as bending or curving applied to a flexible display, apressure change caused by a touch of the user applied to a screen of atouch display of the user device, and a change of a magnetic field orstatic electricity.

Depending on embodiments, the user device may transmit signals includingthe event generated in the user device to the DTV.

Based on the event detected in operation 1010, the user device mayidentify the object of interest of the user among a plurality of regionsor a plurality of objects included in the image content in operation1020. In operation 1020, the user device may identify a region or objectincluded in the image content of a time point when the event isdetected, among the plurality of regions or objects, as the object ofinterest.

In operation 1030, the user device may transmit information on theobject of interest identified in operation 1020 to the DTV so as to beprovided with the additional information corresponding to the object ofinterest.

In response to the transmission of the information on the object ofinterest, the user device may receive the additional informationcorresponding to the object of interest from the DTV in operation 1040.

In operation 1050, the user device may realize the additionalinformation corresponding to the object of interest.

Here, the additional information may include at least one of informationproviding sensory stimulus corresponding to the object of interest,multinational language subtitles related to the image content thatincludes the object of interest, translation of a finger language, aBraille type, oscillation, audio supply, audio conversion, user specificsound, information for enlargement of subtitle font, and the like.

The information providing sensory stimulus corresponding to the objectof interest may include at least one of haptic information stimulating ahaptic sense, sound information stimulating an auditory sensecorresponding to the object of interest, display information stimulatinga visual sense corresponding to the object of interest, smellinformation stimulating an olfactory sense corresponding to the objectof interest, and taste information stimulating a taste sensecorresponding to the object of interest.

FIG. 11 illustrates a DTV 1100 providing user specific interaction,according to example embodiments.

Referring to FIG. 11, the DTV 1100 may include a communication unit1110, a display unit 1130, and a control unit 1150.

The communication unit 1110 may receive image content through a TVnetwork.

The TV network may be understood to include both a broadcasting networkand the Internet.

The display unit 1130 may display the image content received by thecommunication unit 1110.

The control unit 1150 may identify an object of interest of a user amonga plurality of regions or a plurality of objects included in the imagecontent, and provide additional information corresponding to the objectof interest.

The control unit 1150 may include an analysis unit 1151, a determinationunit 1153, a search unit 1155, and a conversion unit 1157.

The analysis unit 1151 may analyze signals received from a user device.Here, the signals received from the user device may include a forceapplied to the user device, a vital signal of the user, preferenceinformation of the user, and the like.

The determination unit 1153 may determine a scene in which signals arechanged among the plurality of regions or the plurality of objectsincluded in the image content, as a result of analysis of the analysisunit 1151.

For example, the determination unit 1153 may determine a content scenein which the force applied to the user device is changed or a contentscene in which an emotion or request of the user is changed.

Here, the control unit 1150 may identify the region or object includedin the scene in which the signals are changed, as the object ofinterest.

The search unit 1155 may search for the additional informationcorresponding to the object of interest.

The search unit 1155 may search for an effect corresponding to thecontent scene determined as the object of interest, or information onemotion or request, that is, the preference information, of the user.

The control unit 1150 may provide the user device with the additionalinformation found by the search unit 1155 as the additional informationcorresponding to the object of interest.

The conversion unit 1157 may convert the additional informationcorresponding to the object of interest, found by the search unit 1155,according to the emotion or request of the user.

FIG. 12 illustrates a DTV 1200 providing user specific interaction,according to example embodiments.

Referring to FIG. 12, the DTV 1200 may include a communication unit1210, a display unit 1230, and a control unit 1250.

The communication unit 1210 may receive image content through a TVnetwork.

The display unit 1230 may display the image content.

The control unit 1250 may be input with selection with respect to theobject of interest among a plurality of regions or a plurality ofobjects included in the image content. In addition, the control unit1250 may provide the additional information corresponding to the objectof interest in response to user interaction with respect to the selectedobject of interest.

FIG. 13 illustrates a user device 1300 providing user specificinteraction, according to example embodiments.

Referring to FIG. 13, the user device 1300 may include a detection unit1310, an identification unit 1320, a communication unit 1330, and arealization unit 1340.

The detection unit 1310 may detect an event generated in the user devicein response to image content displayed through a DTV.

Here, the event may include a change in a vital signal of a userdetected by the user device 1300 and a change in an external forceapplied to the user device. For example, the user device may detect thechange in the vital signal through a device measuring a vital signal,such as an HRM installed inside or outside the user device.

The change in the external force applied to the user device may includedeformation such as bending or curving applied to a flexible display, apressure change caused by a touch of the user applied to a screen of atouch display of the user device, and a change of a magnetic field orstatic electricity.

The identification unit 1320 may identify an object of interest of theuser among a plurality of regions or a plurality of objects included inthe image content base on the event detected by the detection unit 1310.

The communication unit 1330 may transmit information on the object ofinterest or signals including the event to the DTV so as to be providedwith additional information corresponding to the object of interest.

The realization unit 1340 may realize the additional informationcorresponding to the object of interest. The additional information maybe received from the DTV displaying the image content.

FIG. 14 illustrates a system including a DTV 1450 and a plurality ofuser devices which provide user specific interaction, according toexample embodiments.

Referring to FIG. 14, the system providing user specific interaction mayinclude a first user device 1410, a second user device 1420, an N-thuser device 1430, and the DTV 1450.

Each of the user devices may include a detection unit, a communicationunit, and an identification unit which are described in detail withreference to FIG. 13.

The DTV 1450 may include a communication unit 1451, a display unit 1453,an analysis unit 1455, a determination unit 1456, a search unit 1457,and a conversion unit 1458. The analysis unit 1455, the determinationunit 1456, the search unit 1457, and the conversion unit 1458 may bereferred to as a ‘control unit.’

The components of the DTV 1450 are described with reference to FIG. 11.

The above-described embodiments may be recorded in computer-readablemedia including program instructions to implement various operationsembodied by a computer. The media may also include, alone or incombination with the program instructions, data files, data structures,and the like. The program instructions recorded on the media may bethose specially designed and constructed for the purposes ofembodiments, or they may be of the kind well-known and available tothose having skill in the computer software arts. Examples ofcomputer-readable media include magnetic media such as hard disks,floppy disks, and magnetic tape; optical media such as CD ROM disks andDVDs; magneto-optical media such as optical disks; and hardware devicesthat are specially configured to store and perform program instructions,such as read-only memory (ROM), random access memory (RAM), flashmemory, and the like. The computer-readable media may also be adistributed network, so that the program instructions are stored andexecuted in a distributed fashion. The program instructions may beexecuted by one or more processors. The computer-readable media may alsobe embodied in at least one application specific integrated circuit(ASIC) or Field Programmable Gate Array (FPGA), which executes(processes like a processor) program instructions. Examples of programinstructions include both machine code, such as produced by a compiler,and files containing higher level code that may be executed by thecomputer using an interpreter. The above-described devices may beconfigured to act as one or more software modules in order to performthe operations of the above-described embodiments, or vice versa.

Although example embodiments have been shown and described, it would beappreciated by those skilled in the art that changes may be made inthese example embodiments without departing from the principles andspirit of the disclosure, the scope of which is defined in the claimsand their equivalents.

What is claimed is:
 1. A method of providing an interaction in anelectronic device, the method comprising: sensing a user-generatedevent; identifying an object of interest of the user based on the sensedevent; acquiring additional information on the object of interest basedon information on the identified object of interest; and providing theadditional information on the object of interest.
 2. The method of claim1, wherein the acquiring comprises: transmitting the information on theidentified object of interest to an external device; and receiving theadditional information on the object of interest from the externaldevice, wherein the additional information comprises information intowhich the additional information on the object of interest is convertedto be appropriate for the electronic device.
 3. The method of claim 1,wherein the acquiring comprises generating the additional information onthe object of interest by analyzing the information on the identifiedobject of interest.
 4. The method of claim 1, wherein the identifyingcomprises identifying the object of interest among objects or regionsincluded in an image content displayed on an external device differentfrom the electronic device, based on the sensed event.
 5. The method ofclaim 1, wherein the event comprises at least one of a biosignal of theuser sensed through the electronic device and an external force appliedto the electronic device.
 6. The method of claim 1, wherein theidentifying comprises: analyzing the generated event; and identifyingthe object of interest of the user based on a result of the analyzing.7. The method of claim 1, wherein the additional information comprisesat least one of information providing sensory stimulus corresponding tothe object of interest, multinational language subtitles related to animage content that includes the object of interest, translation of afinger language, a Braille type, oscillation, audio supply, audioconversion, user specific sound, and information for enlargement of asubtitle font.
 8. The method of claim 7, wherein the informationproviding sensory stimulus comprises at least one of haptic informationproviding a tactile stimulus corresponding to the object of interest,sound information providing an auditory stimulus corresponding to theobject of interest, display information providing a visual stimuluscorresponding to the object of interest, smell information providing anolfactory stimulus corresponding to the object of interest, and tasteinformation providing a taste stimulus corresponding to the object ofinterest.
 9. The method of claim 1, wherein the identifying comprises:allowing an interaction of the user through the electronic device; andidentifying, as the object of interest, an object or a regioncorresponding to the interaction among objects or regions included in anexternal area of the electronic device.
 10. The method of claim 1,wherein the electronic device comprises at least one of a wearable glassand a screen.
 11. The method of claim 1, wherein the providingcomprises: displaying the additional information on the object ofinterest on a display included in the electronic device; andtransmitting the additional information to an external display device.12. A non-transitory computer-readable medium comprising a program forinstructing a computer to perform the method of claim
 1. 13. A method ofproviding user-specific interaction, the method comprising: displayingan image content on a display device; identifying an object of interestof a user among at least one of regions and objects included in theimage content; and providing additional information corresponding to theobject of interest, wherein the display device analyzes signals of auser device, converts additional information corresponding to theanalyzed signal to be appropriate for the user device, and provides theconverted additional information to the user device, wherein the signalreceived from the user device comprises at least one of a vital signalof the user detected by the user device and an external force applied tothe user device.
 14. The method of claim 13, further comprising:receiving signals from the user device used by the user; wherein theidentifying comprises identifying the object of interest using thesignals received from the user device.
 15. The method of claim 13,further comprising: searching for additional information correspondingto the object of interest.
 16. The method of claim 13, wherein theadditional information corresponding to the object of interest comprisesat least one of information providing sensory stimulus corresponding tothe object of interest, multinational language subtitles related to theimage content that includes the object of interest, translation of afinger language, a Braille type, oscillation, audio supply, audioconversion, user specific sound, and information for enlargement of asubtitle font.
 17. A method of providing user-specific interaction, themethod comprising: displaying an image content on a display device;identifying an object of interest of a user among at least one ofregions and objects included in the image content; and providingadditional information corresponding to the object of interest, whereinthe display device analyzes a signal of a sensor, converts additionalinformation corresponding to the analyzed signal to be appropriate for auser device, and provides the converted additional information to theuser device.
 18. The method of claim 17, wherein the identifyingcomprises: tracking a gaze of the user using the sensor; and identifyingan object tracked by the gaze of the user as the object of interest ofthe user.
 19. The method of claim 17, wherein the identifying comprises:analyzing the signal received from the sensor; determining a scene inwhich the signal is changed in the image content as a result of theanalysis; and identifying a region or object included in the scene inwhich the signal is changed as the object of interest.
 20. The method ofclaim 17, wherein the additional information corresponding to the objectof interest comprises at least one of information providing sensorystimulus corresponding to the object of interest, multinational languagesubtitles related to the image content that includes the object ofinterest, translation of a finger language, a Braille type, oscillation,audio supply, audio conversion, user specific sound, and information forenlargement of a subtitle font.
 21. The method of claim 1, wherein theelectronic device includes a smart phone, a mobile personal computer(PC), wearable glasses, and a flexible display.